Tactile feedback systems and methods for augmented reality and virtual reality systems

ABSTRACT

A tactile feedback system for use with a virtual reality system comprises a support structure and tactile feedback units coupled to and spaced apart around the support structure. A microcontroller is coupled to the tactile feedback units to independently control their activation. The microcontroller has a communication interface configured to communicate with the virtual reality system to receive control signals for controlling operation of the tactile feedback units. The microcontroller activates the tactile feedback units in response to control signals received from the virtual reality system to indicate aspects of the virtual reality experience. In another aspect, a haptic gun for use with a virtual reality system has tactile feedback units and a linear actuator for simulating a recoil of firing the gun. In another aspect a video game system provides a single game play instance for multiple clients utilizing different game platforms.

FIELD OF THE INVENTION

The present invention relates generally to augmented reality and virtualreality systems, and more particularly, to tactile feedback systems andmethods for augmented and virtual reality systems.

BACKGROUND OF THE INVENTION

Virtual reality is a computer technology that replicates an environment,which may be a simulation of a real life environment or an imaginaryenvironment, and simulates a user's presence in the environment allowingthe user to interact with the environment. Current forms of virtualreality are displayed on a computer display or with a special virtualreality headset worn over the user's eyes, which provides visual andsound simulations of the virtual reality experience. Some simulationsalso include additional sensory input, such as haptic/tactile feedbackto the user to simulate various interactions with the virtual reality.The user may interact with the virtual reality using standard computerinput devices such as a keyboard and mouse, and also through multimodaldevices such as gloves or other wearables having sensors to detectmotion and forces, and/or external motion sensors which detect a user'sposition and motions.

Augmented reality is similar to virtual reality in that it involves acomputerized simulation, but differs in that augmented reality utilizesa real-world environment in which certain elements are augmented and/orsupplemented by a computer-generated simulation of sensory input such asvideo, sound, graphics, and/or tactile feedback. The computer-generatedsimulation may be overlaid onto a computer reproduction of thereal-world environment.

For purpose of the present description, the term “virtual reality” shallmean either, or both, virtual reality and augmented reality, as theirdifferences do not affect the present invention.

In order to enhance the virtual reality experience, tactile feedbackdevices, also referred to as haptic devices, have been employed withvirtual reality systems. Tactile feedback devices apply forces,vibrations or motions to a user in order to provide tactile sensoryinput in virtual reality to a user, such as to simulate the sense oftouch or to provide other feedback of experiences within the virtualreality, such as crashing, being shot, loading or shooting a gun, orcolliding with something.

SUMMARY

In one aspect, the present invention is directed to an innovativetactile feedback system for use with a virtual reality headset whichutilizes a plurality of tactile feedback devices to enhance theimmersion into the virtual reality experience and provide entertainingand useful tactile sensory perception to a user within the virtualreality.

Accordingly, in one embodiment, a multi-directional, tactile feedbacksystem for use with a virtual reality headset being used to present avirtual reality experience is provided. The tactile feedback system hasa support structure, a plurality of tactile feedback units mounted tothe support structure in spaced apart relation, and a microcontrollercoupled to the support structure. The support structure may be thehousing and/or other structure of the headset or a separate wearabledevice such as a headband, helmet, cap, other head mounting structure,etc. Each of the tactile feedback units is operably coupled to themicrocontroller and is configured to produce a tactile feedback to auser, and includes a transducer which can apply forces, vibrations ormotions to the user thereby providing tactile sensory input to the user.For instance, the tactile feedback units may be positioned in anangularly spaced array or other pattern around the user's body, such asaround the user's head. The microcontroller is configured to utilizecontrol signals generated during the operation of the headset to controlthe operation of the tactile feedback units. For example, while theheadset is presenting a virtual reality experience, such as playing agame or showing a video, the game or video may generate control signalswhich the microcontroller utilizes to control the operation of thetactile feedback units. In one aspect, the tactile feedback system maybe integrated into the headset such that the microcontroller isintegrated into the headset, in which case the microcontroller may beintegrated with the electronics of the headset or a separate electronicsystem operably coupled to the headset.

In another aspect, the microcontroller may be a separate system having acommunication interface configured to electronically communicate withthe headset, for instance a wireless communication interface, such asBluetooth, Wi-Fi, HaLow or other future wireless standards, or a wiredcommunication interface, such as USB. In this case, the microcontrollerreceives the control signals from the headset via the communicationinterface.

In another aspect, the microcontroller is configured to operate thetactile feedback units in response to the control signals to indicate adirectional aspect of the virtual reality experience. For example, themicrocontroller may activate a specific tactile feedback unit that isangularly located to correspond to the angular location of an eventoccurring in the virtual reality experience, such as the location of ahit (e.g., hit by a shot from a weapon, or punched by another player) onthe user's persona within the virtual reality experience. If the user isshot on the right side, the microcontroller may activate a tactilefeedback unit located on the right side of the angularly spaced array.The control signals may include directional data related to thedirectional aspect of the virtual reality experience being signaledusing the tactile feedback system.

In additional aspects, the tactile feedback system may be configured toprovide a number of use cases. In one use case, the tactile feedbacksystem may be configured to provide a threat detection warning signal tothe user to indicate that the user is being threatened within thevirtual reality experience. This is similar to a “sixth sense” or “SpidySense.” The threat detection warning signal can be a general threatdetection warning signal, or a specific threat detection warning signalsuch as a directional threat detection warning signal by using thedirectional aspect described above. Other specific threat detectionwarning signals include a severity threat detection warning signal toindicate a severity aspect of the virtual reality experience, and animminence/urgency threat detection warning signal to indicate an urgencyaspect of the virtual reality experience. For example, themicrocontroller may activate the tactile feedback system with a level offorce corresponding to a severity level occurring in the virtual realityexperience (e.g., a fender bender versus a head-on crash). Similarly,the microcontroller may activate the tactile feedback system in apattern or other manner corresponding to a degree of urgency occurringin the virtual reality experience. An example of this may be if an enemyis approaching from far away, the tactile feedback units may vibrateslowly or be activated in a slow-rotating pattern, and as the enemyapproaches closer the vibrations and/or speed of the pattern activationincreases.

In another use case, the tactile feedback system may be configured toindicate that a user has been killed in a game, eliminated from a game,or that the user's game has otherwise ended. For instance, themicrocontroller may be configured to activate all of the tactilefeedback units at the same time and/or in a certain manner in responseto a control signal corresponding to such an event.

In still another use case, the tactile feedback system may be configuredto signal to a user the direction and/or intensity of damage to the userwithin the virtual reality experience. The microcontroller may beconfigured to activate particular tactile feedback unit(s) in the arrayrelated to the direction of damage and/or to adjust the intensity of theactivation of such tactile feedback units based on the control signals.

The tactile feedback system may also be configured to signal to the userother aspects of a virtual reality experience, such as vibrationthrusters, various other warnings such as low ammunition, weaponre-loading, weapon upgrading, etc.

Another embodiment of the present invention is directed to methods ofproviding tactile feedback with a virtual reality system being used topresent a virtual reality experience. For example, the methods mayinclude the methods of using the tactile feedback system describedherein, including the additional aspects, features and use cases.

Still another embodiment of the present invention is directed to ahaptic, toy gun game controller for use with a virtual reality headsetfor presenting a virtual reality experience, such as playing a virtualreality game. The haptic gun may be shaped and configured as any type ofgun, such as a handgun, rifle, shotgun, machine gun, laser gun, BB gun,paintball gun, pellet gun, light machine gun (“LMG”), etc. The hapticgun includes a main body having a handle. The body may be in the shapeof the type of gun. A microcontroller having a processor is housed inthe main body. A first communication interface is also housed in themain body. The first communication interface is operably coupled to themicrocontroller and is configured to electronically communicate with theheadset. The first communication interface may be any suitablecommunication interface, including those described herein. The hapticgun has a trigger coupled to the main body, and operably coupled to themicrocontroller. A plurality of tactile feedback units are coupled tothe main body and are spaced apart around the body. Each of the tactilefeedback units is operably coupled to the microcontroller. The tactilefeedback units are configured to produce a haptic feedback to a user,such as the tactile feedback units described above.

A linear actuator is coupled to the main body and operably coupled tothe microcontroller. The linear actuator is configured to provide alinear force simulating a recoil from firing the haptic gun. The hapticgun also has a tracker device housed within the main body. The trackerdevice is configured to provide tracking data to the virtual realitysystem (e.g., a virtual reality headset). For example, the trackerdevice may include accelerometer(s), and/or other sensors to detect themotion and orientation of the haptic gun. The tracker device may have atracker device communication interface, such as a wireless interface orwired interface for communicating with the headset. In other aspects,the tracker device may be integrated with the microcontroller and/or thefirst communication interface.

The microcontroller of the haptic gun is configured to electronicallycommunicate with the headset via the communication interface to receivecontrol signals from the headset for controlling operation of thetactile feedback units and the linear actuator. The microcontrolleroperates the tactile feedback units and the linear actuator based on thecontrol signals. The microcontroller also sends a trigger signal to theheadset in response to actuation of the trigger by the user.

As an example of the operation of the haptic handgun, the user isutilizing the haptic handgun with a virtual reality headset and isplaying a virtual reality game. Within the virtual reality game, theuser sees a target (e.g., an enemy) to shoot at. The user moves and aimsthe haptic gun. The tracker device senses the motion and aim point ofthe gun and provides tracking data to the microcontroller and sends thetracking data to the headset via the tracker device communicationinterface. The virtual reality headset may be configured to utilize thetracking data to control the image of a gun within the virtual realitygame which corresponds to the haptic gun. For instance, as the usermoves and aims the haptic gun, the virtual reality headset may move andaim the gun in the virtual reality game accordingly. When the useractuates the trigger, the microcontroller detects that the trigger hasbeen actuated and sends a trigger signal to the headset. The headsetthen sends control signals to the microcontroller which themicrocontroller uses to control the operation of the tactile feedbackunits and the linear actuator.

In additional aspects, the haptic gun may also be configured to providevarious use cases for controlling the tactile feedback units and thelinear actuator based on the control signals received from the headset.In one use case, the microcontroller may receive a control signal fromthe headset indicating whether to fire in response to sending a triggersignal to the headset. The headset receives the trigger signal anddetermines whether the corresponding gun in the virtual reality game canbe fired (e.g., is there ammunition?; is the gun damaged or otherwiseprevented from firing?). If the headset determines the gun can be fired,the headset sends a “fire gun” control signal to the microcontroller. Inresponse to receiving the fire gun control signal, the microcontrolleris configured to activate the linear actuator to simulate a recoil fromfiring the gun. If the headset determines the gun cannot be fired whenreceiving the trigger signal, the headset can send a “do not fire”control signal, or no control signal at all. In response to receiving ado not fire control signal, the microcontroller may be configured not toactivate the linear actuator, and/or to activate one or more of thetactile feedback units to simulate a hammer or other component of thegun actuating when the gun fails to fire upon pulling the trigger.

In additional use cases, the headset may be configure to send a weaponupgrade control signal, a weapon damage control signal, or other controlsignals. The microcontroller is configured to respond to such signals byactivating the linear actuator and/or tactile feedback units in acertain manner.

Still another embodiment of the present invention is directed to a videogame system for providing a single game play instance in which multipleclients can play, each client utilizing a different game platform. Thevideo game system includes a first game platform executing a virtualreality game on a virtual reality system. The virtual reality system maybe a virtual reality headset, as described herein, or other virtualreality system. The video game system includes a second game platformexecuting a game having a similar representation of the virtual realitygame but modified for the second game platform. In addition, the firstgame platform and second game platform are configured to communicatewith each other to provide a single instance game space for the firstgame platform and second game platform.

In another aspect of the video game system, the first game platform andsecond game platform may be configured to communicate over acommunication network comprising the Internet. In still another aspect,the first game platform may be linked to a third game platform whichdisplays a representation of the virtual reality game on a video displayas the game is being played on the first game platform.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other aspects of embodiments are described in furtherdetail with reference to the accompanying drawings, wherein likereference numerals refer to like elements and the description for likeelements shall be applicable for all described embodiments whereverrelevant:

FIG. 1 is a top view of a multi-directional, tactile feedback system foruse with a virtual reality system, according to one embodiment of thepresent invention;

FIG. 2 is a rear view of the tactile feedback system of FIG. 1;

FIG. 3 is front view of the tactile feedback system of FIG. 1, as wornon the head of a user;

FIG. 4 is a rear view of tactile feedback system of FIG. 1, as worn onthe head of a user;

FIG. 5 is a schematic view of the tactile feedback system of FIG. 1;

FIG. 6 is a schematic view of the tactile feedback system of FIG. 1 incombination with a virtual reality headset for presenting a virtualreality experience;

FIG. 7 is a diagram depicting a directional damage scenario beingsignaled by the tactile feedback system of FIG. 1;

FIG. 8 is a flow chart illustrating a method of operation of the tactilefeedback system of FIG. 1 for the directional damage scenario of FIG. 7,according to another embodiment of the present invention;

FIG. 9 is a schematic depicting a direction and damage intensity beingsignaled by the tactile feedback system of FIG. 1;

FIG. 10 is a flow chart illustrating a method of operation of thetactile feedback system of FIG. 1 for the direction and damage intensityscenario of FIG. 9, according to another embodiment of the presentinvention;

FIG. 11 is a schematic depicting a general threat detection warningscenario being signaled by the tactile feedback system of FIG. 1;

FIG. 12 is a flow chart illustrating a method of operation of thetactile feedback system of FIG. 1 for the general threat detectionwarning scenario of FIG. 11, according to another embodiment of thepresent invention;

FIG. 13 is a schematic depicting a directional threat detection warningscenario being signaled by the tactile feedback system of FIG. 1;

FIG. 14 is a flow chart illustrating a method of operation of thetactile feedback system of FIG. 1 for the directional threat detectionwarning scenario, according to another embodiment of the presentinvention;

FIG. 15 is a side view of a haptic gun, according to another embodimentof the present invention;

FIG. 16 is a side, cut-away view of the haptic gun of FIG. 15;

FIG. 17 is a high-level schematic view of the electronic system for thehaptic gun of FIG. 15, according to another embodiment of the presentinvention;

FIG. 18 is a schematic view of the electronic system for the linearactuator of the haptic gun of FIG. 15, according to another embodimentof the present invention;

FIG. 19 is a is a flow chart illustrating a method of operation of thehaptic gun of FIG. 15, according to another embodiment of the presentinvention;

FIG. 20 is a flow chart illustrating a method of operation of the hapticgun of FIG. 15 for a trigger actuation scenario, according to anotherembodiment of the present invention;

FIG. 21 is a schematic view of a virtual reality system for providing asingle game instance for multiple clients each utilizing a differentgame platform.

DETAILED DESCRIPTION

Referring to FIGS. 1-6, one embodiment of a multi-directional, tactilefeedback system 10 for use with a virtual reality system 30 (see FIG. 6,in which the virtual reality system 30 is a virtual reality headset 30)configured to present a virtual reality experience to a user isillustrated. The virtual reality system 30 may be any suitable systemfor presenting a virtual reality experience, such as a virtual realityheadset, goggles, helmet, video display(s), etc. In general, the virtualreality system 30 includes a processor (e.g., a microprocessor), asoftware program configured to program the virtual reality system topresent the virtual reality experience, a display device (e.g., videodisplay(s)), and a system communication interface. The virtual realityexperience can be any type of entertainment or informational program,such as a video game, a movie, a video, etc. As explained above, theterm “virtual reality” as used herein includes either, or both, virtualreality and augmented reality. The system communication interface may beany suitable communication interface for communicating with the tactilefeedback system 10 and/or other electronic systems. For instance, theinterface can be a wireless communication interface, such as Bluetoothor Wi-Fi, or a wired communication interface, such as USB.

Turning to FIGS. 1 and 2, the tactile feedback system 10 includes asupport structure 12, which in this case is a headband 12 configured tobe worn on the head 14 of a user 11 (see FIGS. 3 and 4). As explainedherein, the support structure 12 may be integrated with a virtualreality system 30 (e.g., integrated with a virtual reality headset), orit may be a separate wearable device such as the headband 12, or otherwearable device such as a hat, helmet, goggles, glasses, etc. In thisdescribed embodiment, the tactile feedback system 10 is a separatewearable device which communicates with a virtual reality system 30,such as the virtual reality headset 30 as shown in FIG. 6. A pluralityof tactile feedback units 16 are mounted to the headband 12 angularlyspaced apart around the headband 12. Each of the tactile feedback units16 is a tactile feedback device which can apply forces, vibrations, ormotions which provide tactile sensory input to the user 11. For example,the tactile feedback units 16 may comprise a vibration motor forapplying a vibration feedback to the user 11, such as a 10 mm-3-4.5 voltvibration motor. Each of the tactile feedback units 16 is operablycoupled to a microcontroller 20 (described below) such that each tactilefeedback unit 16 can be operated independently of the other tactilefeedback units 16. In this described embodiment, the feedback system 10has eight tactile feedback units 16 which are angularly spaced at about45 degrees from each other. More or less tactile feedback units 16 mayutilized, and may be spaced about equal angles around the headband, orthey may be spaced unequally, or even spaced in multiple groups. Forinstance, the feedback system 10 may have from 1 to 30, or more, tactilefeedback units 16, such as four units 16 (e.g., spaced at about 90degrees), 5 units 16 (e.g., spaced at about 72 degrees), 6 units 16(e.g., spaced at about 60 degrees), 10 units (e.g., spaced at about 36degrees), etc.

A control module 18 housing the microcontroller 20 having a processor,user input controls 22, and a power supply 24 (which may include abattery or other power source) is also mounted to the headband 12.Turning to FIGS. 5 and 6, the control module 18 includes an electroniccircuit 26, such as a printed circuit board or integrated circuit,operably connecting the microcontroller 20, the power supply 24, and theuser input controls 22. The control module 18 also operably connectseach of the tactile feedback units 16 to the microcontroller 20 and thepower supply 24 in order to control and power the actuation of thetactile feedback units 16. The control module 18 also has a USBcommunication interface 28, which may be integrated into themicrocontroller 20, or separate from the microcontroller 20 and operablycoupled to the microcontroller 20. The USB communication interface 28 isconfigured to provide electronic communication between microcontroller20 and the virtual reality system 30. The tactile feedback system 10also has a wireless communication interface 32, which may be integratedinto the microcontroller 20, or separate from the microcontroller 20 andoperably coupled to the microcontroller 20. The wireless communicationinterface 32 is configured to provide electronic communication betweenmicrocontroller 20 and the virtual reality system 30 and the one or morecommunication interfaces. In use, the tactile feedback system 10 may beoperably connected to the virtual reality system 30 using either, orboth, of the USB communication interface 28 and the wirelesscommunication interface 32, to receive control signals from the virtualreality system 30 which the microcontroller 20 uses to control theoperation of the tactile feedback units 16.

The user input controls 22 are operably coupled to the microcontroller20 for adjusting the baseline activation intensity (e.g., vibrationintensity) of the tactile feedback system 10. The baseline activationintensity is an intensity of activation from which the different levelsof intensity at which the tactile feedback units 16 are activated forvarious tactile signals to the user, as further described below. Theuser input controls 22 may comprise a pair of push buttons operablycoupled to the microcontroller 20, or any other suitable controls forproviding user input to the microcontroller 20 to adjust the baselineactivation intensity. The microcontroller 20 is configured to adjust thebaseline activation intensity at which the microcontroller 20 activatesthe tactile feedback units 16 in response to the user input controls 22.

Turning to FIGS. 7 and 8, a diagram and flowchart depict the tactilefeedback system 10 configured to perform a method 100 for signaling adirectional damage scenario. As shown in FIG. 7, the tactile feedbacksystem 10 is configured to activate the tactile feedback unit 16 a toindicate to the user 11 the approximate direction of damage sustained bythe user 11 in the virtual reality experience. For instance, in theexample depicted in FIG. 7, the user 11 is damaged (e.g., shot) in thefront/left, such that the tactile feedback system 10 activates thetactile feedback unit 16 a located on the front/left of the tactilefeedback system 10. As shown in the diagram, the user 11 turns to hisleft in response to the activation of the tactile feedback unit 16 a.

Turning to FIG. 8, the method 100 comprises a step 102, in which anevent occurs at an angular location relative to the user 11 in thevirtual reality experience being executed on the virtual reality system30. In the example of FIG. 7, the event is the user 11 being shot in thefront/left of the user 11. At step 104, the virtual reality system 30sends a direction control signal to the microcontroller 20 via the USBcommunication interface 28 or the wireless communication interface 32.The direction control signal indicates the angular direction of theevent. For instance, the control signals may be coded and themicrocontroller 20 is configured to relate the coded signal to anangular direction. At step 106, the microcontroller 20 processes thedirection control signal to determine the tactile feedback unit 16 acorresponding to the angular location of the event. For example, thecorresponding tactile feedback unit 16 may be the tactile feedback unitwhich best represents the direction of the event. Then, at step 108, themicrocontroller 20 activates the tactile feedback unit 16 a based on thedirection control signal corresponding to the direction of the eventrelative to the location of the user 11 in the virtual realityexperience.

FIGS. 9 and 10 illustrate another use case for the tactile feedbacksystem 10 performing a method 120 to signal to the user the direction ofdamage and damage intensity incurred by the user 11 in the virtualreality experience. The angular location of the damage is basically thesame as described above for the method 100. However, as shown in FIG. 9,instead of activating just the one tactile feedback unit 16 a, themicrocontroller 20 also activates one or more adjacent tactile feedbackunits 16 b and 16 c at a lower intensity than it activates the maintactile feedback unit 16 a. The number of adjacent tactile feedbackunits 16 and the intensity of the activation of the adjacent tactilefeedback units 16 depend on the level of intensity of the damage. Forinstance, for minimal damage, perhaps no adjacent tactile feedback units16 are activated. Above a first threshold and below a second threshold,perhaps only the two adjacent tactile feedback units 16 b and 16 c areactivated at a very low intensity. As the level of intensity increasesfrom the first threshold to the second threshold, the intensity of thetwo adjacent tactile feedback units 16 b and 16 c may be increased, butstill lower than the intensity of the main tactile feedback unit 16 a inorder to still effectively signal the direction of the damage. When thesecond threshold is reached, three or four adjacent tactile feedbackunits 16 may be activated, with increasing intensity based on anincrease in the intensity of damage, and so on as the intensity of thedamage increases.

Accordingly, turning to the flow chart of FIG. 10, the method 120comprises step 122, in which damage occurs to the user 11 in the virtualreality experience being executed on the virtual reality system 30. Thedamage has an associated direction of damage and intensity of damage. Atstep 124, the virtual reality system 30 sends a damage direction anddamage intensity control signal (this may be a single signal or aseparate signal for damage direction and damage intensity) to themicrocontroller 20 via the USB communication interface 28 or thewireless communication interface 32, indicating the direction of thedamage and the intensity of the damage. At step 126, the microcontroller20 processes the damage direction and damage intensity control signal(s)to determine the main tactile feedback unit 16 a corresponding to theangular location of the event and the adjacent tactile feedback unit(s)16 b and 16 c to be activated and their respective intensity. Then, atstep 108, the microcontroller 20 activates the tactile feedback units 16a, 16 b and 16 c based on the control signal to indicate the directionand intensity of the damage occurring in the virtual reality experience.

Turning to FIGS. 11 and 12, a diagram and flowchart depict the tactilefeedback system 10 configured to perform a method 130 for signaling ageneral threat detection warning. A general threat detection warningmeans it is non-directional, i.e., it does not indicate to the user adirection of a threat (e.g., potential danger) to the user occurring inthe virtual reality experience. A general threat detection warning alsodoes not necessarily indicate other specific characteristics of thethreat, such as imminence/urgency or severity. As shown in the diagramof FIG. 11, the arrow 131 shows that the threat detection warning signalmay be performed by sequentially activating each of the tactile feedbackunits 16 (individually or in groups) for a short duration or pulse in arotational sequence around the array. The arrow 131 shows the tactilefeedback units 16 being activated in a counter-clockwise sequence, butthe sequence could also be clockwise, or a counter-clockwise sequencecould signal a first type of threat while a clockwise sequence couldsignal a second type of threat.

Referring to FIG. 12, the method 130 comprises step 132 in which athreat detection warning event occurs in the virtual reality experiencebeing executed on the virtual reality system 30. At step 134, thevirtual reality system 30 sends a general threat detection warningcontrol signal to the microcontroller 20 via the USB communicationinterface 28 or the wireless communication interface 32. At step 136,the microcontroller 20 processes the general threat detection warningcontrol signal to determine the sequence for activating the tactilefeedback units 16 corresponding to the control signal. At step 138, themicrocontroller 20 activates the tactile feedback units 16 (e.g., in asequence) to signal a general threat detection warning, as describedabove.

Referring now to FIGS. 13 and 14, a diagram and flowchart depict thetactile feedback system 10 configured to perform a method 140 forsignaling a directional threat detection warning. A directional threatdetection warning indicates both that there is a threat (e.g., apotential danger) and the direction of the threat event relative to theuser 11 occurring in the virtual reality experience. As shown in FIG.13, the tactile feedback system 10 is configured to activate the tactilefeedback unit 16 a to indicate to the user 11 the approximate angulardirection of the threat to the user 11 in the virtual realityexperience. To indicate a threat, as opposed to damage for example, themicrocontroller 20 may be configured to activate the tactile feedbackunit 16 a by oscillating or pulsing (i.e., quickly activating andde-activating) the tactile feedback unit 16 a multiple times) thetactile feedback unit 16 a. For instance, in the example depicted inFIG. 13, the threat is located in the front/left of the user 11, suchthat the tactile feedback system 10 activates the tactile feedback unit16 a located on the front/left of the tactile feedback system 10. Asimilar method to method 140 may also be utilized to provide adirectional signal to a user for events, items or situations other thanthreats, such as for providing a directional indicator to signal anevent, item or situation in a particular direction within the virtualreality experience.

As shown in the flow chart of FIG. 14, the method 140 comprises step 142in which a threat detection warning event occurs at an angular locationrelative to the user 11 in the virtual reality experience being executedon the virtual reality system 30. At step 144, the virtual realitysystem 30 sends a directional threat detection warning control signal tothe microcontroller 20 via the USB communication interface 28 or thewireless communication interface 32. The directional threat detectionwarning control signal indicates the angular direction of the threatevent. At step 146, the microcontroller 20 processes the directionalthreat detection warning control signal to determine the tactilefeedback unit 16 corresponding to the angular location of the threatevent. At step 148, the microcontroller 20 activates the tactilefeedback unit 16 a corresponding to the direction of the threat eventrelative to the user 11 in the virtual reality experience by, e.g.,oscillating or pulsing the tactile feedback unit 16 a, as describedabove.

Referring now to FIGS. 15-18, another embodiment of the presentinvention is directed to a haptic, toy gun, game controller 150 for usewith a virtual reality system 30. The virtual reality system 30 may bethe same, or similar to the virtual reality system described above forthe tactile feedback system 10. As shown in FIG. 15, the haptic gun 150is shaped like a handgun and has a main body 152 having a handle 154.Although the exemplary embodiment shown in FIGS. 15-16 is in the shapeof a handgun, the gun may be shaped and configured as any suitable typeof gun, including without limitation, a handgun, rifle, shotgun, machinegun, laser gun, BB gun, paintball gun, pellet gun, light machine gun,replica of an actual gun, or any made-up gun. Thus, the body 152 is inthe shape of the desired gun.

Turning to FIG. 16, a plurality of tactile feedback units 16 are coupledto the main body 152, such as being housed in the main body 152. Thetactile feedback units 16 may be the same as, or similar to, to thetactile feedback units 16 described above with respect to the tactilefeedback system 10. The tactile feedback units 16 are located spacedapart within the main body 152, such as a tactile feedback unit 16located in the handle 154, a tactile feedback unit 16 located in themiddle of the main body 152, and a tactile feedback unit 16 located nearthe front end of the main body 152 (e.g., proximate the end of thebarrel of the gun 150). More or fewer tactile feedback units 16 may beutilized, depending on the desired effects to be produced by the tactilefeedback units 16.

A linear actuator 156 is also coupled to the main body 152, in this casehoused within the main body 152. The linear actuator 156 is configuredto be activated to provide a linear force simulating a recoil fromfiring the haptic gun 150. The linear actuator 156 may also provide alinear force in the opposite direction when the linear actuator is resetwhich can simulate the gun 150 loading another round of ammunition.

A microcontroller 20 having a processor, a tracker device 158 and apower source 160 are also housed in the main body 152. Themicrocontroller 20 may be the same as, or similar to, to themicrocontroller 20 described above with respect to the tactile feedbacksystem 10. The haptic gun 150 has a gun communication interface 166which may be integrated into the microcontroller 20, or may be aseparate component operably coupled to the microcontroller 20. The guncommunication interface 166 may be, for example, a wired communicationinterface such as USB, or a wireless communication interface such asBluetooth or Wi-Fi, and is configured to provide electroniccommunication between the microcontroller 20 and system communicationinterface of the virtual reality system 30.

Each of the tactile feedback units 16 and the linear actuator 156 areoperably coupled to the microcontroller 20 such that each of the tactilefeedback units 16 and the linear actuator 156 may be activatedindependently of each other. A trigger 162 is also coupled to the mainbody 152. The trigger 162 is configured to be actuated by pulling thetrigger with a finger of a user. The trigger 162 is also operablycoupled to the microcontroller 20.

The tracker device 158 is configured to provide tracking data to thevirtual reality system 30. The tracker device 158 has a trackercommunication interface, such as a wireless communication interface or aUSB communication interface, for communicating the tracking data to thesystem communication interface of the virtual reality system 30. Thetracker device 158 may have accelerometer(s), and/or other sensors todetect the motion, location, and/or orientation of the haptic gun 150.

Turning to FIG. 17, a high-level schematic of the electronic system ofthe haptic gun 150 is shown. It is understood that the schematic of FIG.17 does not include all of the components of the electronic system(e.g., transistors, diodes, resistors, etc.), but one or ordinary skillin the art is enabled to incorporate the high-level schematic into anoperable electronic system. The electronic system includes an electroniccircuit 164 operably interconnecting the microcontroller 20, the powersource 160, the tracker device 158, the tactile feedback units 16 andthe linear actuation 156. Referring to FIG. 18, a schematic is shown foran exemplary dual power relay 168 for powering the linear actuator 156.The dual power relay 168 includes two 5 volt power relays 170 which areoperably coupled to the microcontroller 20 and to the linear actuator156.

Turning to FIG. 19, a method 200 for operating the haptic gun 150 whileusing the haptic gun 150 with a virtual reality system 30 is shown. Thehaptic gun 150 is linked to the virtual reality system 30 such that themicrocontroller 20 and virtual reality system may electronicallycommunicate via the gun communication interface 166 and the systemcommunication interface. At step 202, the virtual reality system 30displays a target in the virtual reality experience which is seen by theuser. At step 204, the user moves and aims the haptic gun 150 to aim atthe target, and the tracker device 158 senses the motion and aim pointof the haptic gun 150. At step 206, the tracker device 158 sendstracking data to the virtual reality system 30 via the trackercommunication interface and the system communication interface. At step208, the virtual reality system 30 controls an image of a gun within thevirtual reality experience corresponding to the haptic gun 150, anddisplays the image to the user based on the tracking data. At step 210,the microcontroller 20 detects whether that the trigger 162 has beenactuated. If the user actuates the trigger 162, the microcontroller 20detects that the trigger has been actuated, and at step 212, themicrocontroller 20 sends a trigger signal to the virtual reality system30. At step 214, the virtual reality system 30 processes the triggersignal and determines a control signal or signals to send to themicrocontroller 20 for controlling the operation of the tactile feedbackunits 16 and/or the linear actuator 156. At step 216, the virtualreality system 30 sends the control signal(s) to the microcontroller 20.At step 218, the microcontroller 20 processes the control signal(s) todetermine if and how to actuate the tactile feedback unit(s) 16 and/orthe linear actuator 156. At step 220, the microcontroller activates thetactile feedback unit(s) 16 and/or the linear actuator 156 based on thecontrol signal(s). If the trigger has not been actuated at step 210,then at step 222, the virtual reality system determines a control signalbased on the virtual reality experience. For example, the correspondinggun in the virtual reality experience may be hit or damaged. At step224, the virtual reality system 30 sends a control signal to themicrocontroller 20. Then, at step 226, the microcontroller 20 processesthe control signal to determine if and how to actuate the tactilefeedback unit(s) 16 and/or the linear actuator 156. At step 228, themicrocontroller activates the tactile feedback unit(s) 16 and/or thelinear actuator 156 based on the control signal(s).

Turning to the flow chart of FIG. 20, a method 230 for a particular usecase for firing the haptic gun 150 is shown. The method 230 may beperformed following step 212 and replaces the steps following 212 in themethod 200 described above and shown in FIG. 19. At step 232, thevirtual reality system 30 determines whether a virtual gun within thevirtual reality experience corresponding to the haptic gun 150 hasammunition. If the virtual gun does not have ammunition, then at step234, the virtual reality system 30 sends a “No Ammo” control signal tothe microcontroller 20. At step 236, the microcontroller processes the“No Ammo” control signal to determine if and how to activate the tactilefeedback units 16 and/or the linear actuator 156. If the microcontroller20 is configured (i.e., programmed) to activate the tactile feedbackunits 16 and/or the linear actuator 156 in response to a “No Ammo”control signal, then at step 238, the microcontroller 20 activates thetactile feedback units 16 and/or the linear actuator 156. Typically, ina “No Ammo” scenario, the microcontroller 20 does not activate thelinear actuator 156, but may activate one or more of the tactilefeedback units 16 to simulate dry fire (e.g., to simulate a hammer orfiring pin being actuated, but no round fired).

If the virtual reality system 30 determines at step 232 that the gun hasammunition to be fired, then at step 240, the virtual reality system 30determines if the virtual gun is damaged or cannot fire for any otherreason within the virtual reality experience. If the gun is damaged orcannot fire, at step 242, the virtual reality system sends a “gundamaged” or “gun inoperable” control signal to the microcontroller 20.At step 244, the microcontroller 20 processes the control signal todetermine if and how to activate the tactile feedback units 16 and/orthe linear actuator 156. If the microcontroller 20 is configured (i.e.,programmed) to activate the tactile feedback units 16 and/or the linearactuator 156 in response to a “gun damaged” or “gun inoperable” controlsignal, then at step 246, the microcontroller 20 activates the tactilefeedback units 16 and/or the linear actuator 156. Typically, in a “gundamaged” or “gun inoperable” scenario, the microcontroller 20 does notactivate the linear actuator 156, but may activate one or more of thetactile feedback units 16 to simulate dry fire (e.g., to simulate afailed attempt to fire the gun).

If the virtual reality system 30 determines at step 240 that the gun isnot damaged or otherwise inoperable, then at step 248, the virtualreality system 30 sends a “fire gun” control signal to themicrocontroller 20. At step 250, the microcontroller 20 processes the“fire gun” control signal to determine if and how to activate thetactile feedback units 16 and/or the linear actuator 156, or to simplyexecute a pre-programmed “firing sequence.” Typically, in a “fire gun”scenario, the microcontroller 20 is configured (i.e., programmed) toactivate the linear actuator 156 in response to a “fire gun” controlsignal. The microcontroller 20 may also activate one or more of thetactile feedback units 16 to indicate certain scenarios, such as firinga very powerful gun, or other scenarios. At step 252, themicrocontroller 20 activates the linear actuator 156 and/or tactilefeedback units 16, based on the “fire gun” control signal.

In another aspect, the virtual reality system 30 may send a controlsignal to the microcontroller 20 indicating how many rounds ofammunition the gun has to fire, and the microcontroller 20 is configuredto perform the firing sequence each time the trigger is actuated (orheld depressed in the case of a simulated automatic gun) for the numberof rounds of ammunition remaining. When the number of rounds has beenexhausted, the microcontroller 20 receives another control signal fromthe virtual reality system in order to perform any further firingsequences. This aspect can account for communication delays between thehaptic gun 150 and the virtual reality system 20, such as delays causedby EMI or other communication interference.

Turning to FIG. 21, another embodiment of the present invention isdirected to a video game system 300 for providing a single game playinstance wherein multiple clients can play the same game instance witheach client utilizing a different game platform. The video game system300 includes a first game platform 302 which is a virtual reality system30, as described herein. The first game platform 302 executes a virtualreality game 304 on the virtual reality system 30. The virtual realitysystem 30 may be a virtual reality headset, as described herein, orother virtual reality system. The video game system 300 includes asecond game platform 306 executing a game 308 having a similarrepresentation of the virtual reality game but modified for the secondgame platform 306. The first game platform 302 and second game platform306 each have communication interfaces configured to allow them tocommunicate with each other to provide a single instance game space forthe first game platform 302 and second game platform 306. For example,the communication interfaces may be wired interfaces, or wirelessinterfaces, as described herein. Furthermore, the first game platform302 and second game platform 306 may communicate with each other overthe Internet 310 using their respective communication interfaces toconnect to the Internet 310.

In another aspect, the video game system 300 may also include a thirdgame platform 312 which is in communication with the first game platform302. The first game platform 302 and third game platform 312 areconfigured to allow the third game platform 312 to display arepresentation of the virtual reality game 304 on a video display as itis being played on the first game platform 302. For example, the thirdgame platform 312 may be a tablet computer, a smart TV, a personalcomputer, a smart phone, or other computing device not having virtualreality capability, such that a spectator can watch the game play whilea user is using the virtual reality game 304. For instance, if the useris using a virtual reality headset, only the user can see the game play,so it is useful to allow others to view the game play on the third gameplatform 312. It is understood, that additional game platforms, same,similar, or different to the first game platform 202 or the second gameplatform 306, can be connected and added to the video game system 300 toallow additional clients to play the same single game instance.Similarly, additional game platforms, same, similar, or different to thethird game platform 312, may be connected to the video game system 300to allow additional observers to view the game play. Also, additionalgame platforms 302 may be connected to the video game system 300. In thecase of additional game platforms 302, specific game platforms 312 canbe slaved to various headsets, and/or games platforms 312 can be allowedto switch from one game platform 302 to another game platform 302 toview the game from different views from their perspective. In additionwhen a client's game ends, e.g., the client's character is dead or theclient's active participation in the game otherwise terminates, thesystem 300 allows such client to continue to view the gamer. Forinstance, the system 300 may allow such client to view the game from theperspective of one of the other clients that is still active in thegame. Any of the game platforms 302 and 306 may also be configured toregister a client's death and/or termination of such client's activeparticipation in the game, for example, as a result of a “gameconclusion” event. The game platform may send a “end game” signal to allof the other game platforms involved in the single game instance whichindicates to that the respective client is dead or has terminated activeparticipation in the game.

In some embodiments, observers viewing a game through a game platformconnected to the video game system 300 may participate in or interactwith game play through certain predefined audience actions, such ascheering or booing, wagering on a game play result or event, conversingor chatting with one or more players or observers, deploying game playobjects into game play, or any other game play-related action.Additional non-limiting examples of audience actions may include causingan attack/defense in a combat-themed game (e.g., throwing a grenade,firing a shot, launching a missile, casting a spell into the game);helping or obstructing a player (e.g., giving/taking health, mana,ammunition, virtual currency, weapons, power-ups, etc.); giving hints toplayers of the game; wagering with other observers/players on theoutcome of the game; cheering on a particular player.

Although particular embodiments have been shown and described, it is tobe understood that the above description is not intended to limit thescope of these embodiments. While embodiments and variations of the manyaspects of the invention have been disclosed and described herein, suchdisclosure is provided for purposes of explanation and illustrationonly. Thus, various changes and modifications may be made withoutdeparting from the scope of the claims. For example, not all of thecomponents described in the embodiments are necessary, and the inventionmay include any suitable combinations of the described components, andthe general shapes and relative sizes of the components of the inventionmay be modified. Accordingly, embodiments are intended to exemplifyalternatives, modifications, and equivalents that may fall within thescope of the claims. The invention, therefore, should not be limited,except to the following claims, and their equivalents.

What is claimed is:
 1. A haptic gun for use with a virtual realitysystem configured to present a virtual reality experience, comprising: amain body having a handle; a microcontroller housed in the main body,the microcontroller comprising a processor; a communication interfacehoused in the main body, the communication interface operably coupled tothe microcontroller and configured to electronically communicate withthe virtual reality system; a trigger coupled to the main body, andoperably coupled to the microcontroller; a plurality of tactile feedbackunits coupled to the main body, each of the tactile feedback unitsoperably coupled to the microcontroller and configured to produce ahaptic feedback to a user; a linear actuator coupled to the main body,the linear actuator operably coupled to the microcontroller andconfigured to provide a linear force simulating a recoil from firing thegun; and a tracker device housed within the main body, the trackerdevice configured to provide tracking data to the virtual reality systemrepresentative of a motion and an aim point of the gun; and themicrocontroller is configured to: electronically communicate with thevirtual reality system via the communication interface includingreceiving control signals from the virtual reality system forcontrolling operation of the tactile feedback units and the linearactuator; the control signals including at least two different weaponcondition control signals, each weapon condition control signalcorresponding to a different weapon condition event occurring in avirtual reality experience, the weapon condition events being eventsrelating to the condition of the weapon, the at least two weaponcondition events selected from the group consisting of (a) weapon isdamaged (b) weapon is upgraded, and (c) weapon is out of ammunition;operate the tactile feedback units and the linear actuator based on thecontrol signals; and send a trigger signal to the virtual reality systemin response to actuation of the trigger; and, in response to receiving arespective one of the weapon condition control signals, selectivelyactuate one or more of the tactile feedback units in a predeterminedsequence corresponding to the respective weapon condition controlsignal, wherein the predetermined sequence corresponding to the “weaponis damaged” event is a weapon damaged sequence, the predeterminedsequence corresponding to the “weapon is upgraded” event is a weaponupgrade sequence, and the predetermined sequence corresponding to the“weapon is out of ammunition” is a weapon out of ammo sequence.
 2. Thehaptic gun of claim 1, wherein the tracker device comprises a trackercommunication interface.
 3. The haptic gun of claim 1, wherein themicrocontroller comprises the communication interface.
 4. The haptic gunof claim 1, wherein the communication interface is a wirelesscommunication interface operating on a wireless communication protocol.5. The haptic gun of claim 1, wherein the weapon out of ammo sequencesimulates a hammer or firing pin being actuated.
 6. The haptic gun ofclaim 1, wherein the weapon out of ammo sequence simulates a failedattempt to fire a gun.
 7. The haptic gun of claim 1, wherein the weaponout of ammo sequence simulates a hammer or firing pin being actuated andthe weapon out of ammo sequence simulates a failed attempt to fire agun.
 8. A system for providing a virtual reality experience utilizing ahaptic gun controller, comprising: a virtual reality system comprising aprocessor, a software program, a display device, and a systemcommunication interface, and configured to present a virtual realityexperience to a user; and a haptic gun controller comprising: a mainbody having a handle; a microcontroller housed in the main body, themicrocontroller comprising a processor and a gun communicationinterface; a trigger coupled to the main body, and operably coupled tothe microcontroller; a plurality of tactile feedback units coupled tothe main body, each of the tactile feedback units operably coupled tothe microcontroller and configured to produce a haptic feedback to auser; a linear actuator coupled to the main body, the linear actuatoroperably coupled to the microcontroller and configured to provide alinear force simulating a recoil from firing the gun; and a trackerdevice housed within the main body, the tracker device operably coupledto the microcontroller, the tracker device configured to providetracking data to the virtual reality system representative of a motionand an aim point of the gun; wherein the microcontroller is configuredto: electronically communicate with the virtual reality system via thecommunication interface including receiving control signals from thevirtual reality system for controlling operation of the tactile feedbackunits and the linear actuator; the control signals including at leasttwo different weapon condition control signals, each weapon conditioncontrol signal corresponding to a different weapon condition eventoccurring in a virtual reality experience, the weapon condition eventsbeing events relating to the condition of the weapon, the at least twoweapon condition events selected from the group consisting of (a) weaponis damaged, (b) weapon is upgraded, and (c) weapon is out of ammunition;operate the tactile feedback units and the linear actuator based on thecontrol signals; and send a trigger signal to the virtual reality systemin response to actuation of the trigger; and, in response to receiving arespective one of the weapon condition control signals, selectivelyactuate one or more of the tactile feedback units in a predeterminedsequence corresponding to the respective weapon condition controlsignal, wherein the predetermined sequence corresponding to the “weaponis damaged” event is a weapon damaged sequence, the predeterminedsequence corresponding to the “weapon is upgraded” event is a weaponupgrade sequence, and the predetermined sequence corresponding to the“weapon is out of ammunition” is a weapon out of ammo sequence; whereinthe virtual reality system is configured to: communicate with the guncommunication interface via the system communication interface; receivethe trigger signal from the haptic gun; send the control signals to thehaptic gun; receive the tracking data; and utilize the trigger signaland tracking data to control a virtual gun within the virtual realityexperience corresponding to the haptic gun.
 9. The system of claim 8,wherein the virtual reality system is further configured such that uponreceiving a trigger signal from the gun that the trigger has beenactivated, determine whether a virtual gun within the virtual realityexperience has ammunition, and (i) when determining the virtual gun doesnot have ammunition, send a control signal to the haptic gun that theammunition is empty (not to fire); (ii) when determining that thevirtual gun has ammunition, send a firing sequence control signal to thehaptic gun; and wherein the haptic gun controller is further configuredto activate the linear actuator to simulate a recoil from firing the gunand to reset the linear actuator, both in response to receiving thefiring sequence control signal.
 10. The system of claim 9, wherein oneof the weapon condition control signals is a weapon upgrade controlsignal corresponding to a weapon upgrade event in the virtual realityexperience and the virtual reality system is further configured to sendthe weapon upgrade control signal to the haptic gun controller; andwherein the predetermined sequence corresponding to the weapon upgradecontrol signal is a weapon upgrade sequence, and the haptic guncontroller is further configured to activate one or more of the tactilefeedback units and/or the linear actuator in the weapon upgrade sequencein response to receiving the weapon upgrade control signal.
 11. Thesystem of claim 8, wherein one of the weapon condition control signalsis a weapon damage control signal corresponding to a weapon damage eventin the virtual reality experience and the virtual reality system isfurther configured to send the weapon damage control signal to thehaptic gun controller; and wherein the haptic gun controller is furtherconfigured to activate one or more of the tactile feedback units and/orthe linear actuator in a weapon damage sequence in response to receivingthe weapon damage control signal.
 12. The system of claim 8, wherein theweapon out of ammo sequence simulates a hammer or firing pin beingactuated.
 13. The system of claim 8, wherein the weapon out of ammosequence simulates a failed attempt to fire a gun.
 14. The system ofclaim 8, wherein the weapon out of ammo sequence simulates a hammer orfiring pin being actuated and the weapon out of ammo sequence simulatesa failed attempt to fire a gun.